Virtual machine (VM) hypervisor programs have been in public use for many years, e.g. IBM VM/System Product (SP). VM/SP normally is loaded into the high address end of main storage and coordinates the running of programs by a plurality of users who respectively interface a data processing system from any of a number of keyboard/display terminals which are usually distant from the one or more central processing units (CPUs) and the main storage (MS) of the system.
The advantage of VM is that it gives each of its users the apparent data processing power of a large system by giving each user an apparent or logical CPU. Plural logical CPUs could share each real CPU resource(s) in the system. The VM users (who are sometimes called "guests" of the VM "host" control program) are assigned by the VM control program to different MS areas in which to operate as the user performs work on the system.
Any VM guest may run any type of program that is designed to interface the architecture of the connected real system, e.g. the S/360, S/370 or S/370XA architected instruction set may be used by any guest program running on a S/370XA hardware system.
Operating programming systems, such as MVS, have been run as one of plural guests under VM/SP, under which MVS/SP was often the preferred guest because it was used as the production system. The preferred guest managed its assigned part of MS, which is called the guest's real storage since to the guest operating system this is the system storage under its control. The preferred guest was run with its virtual storage addresses equal to and translating to the host's real storage addresses, and was sometimes called the V=R guest.
Other simultaneous guests under VM operation were, however, virtual guests (sometimes called V=V guests) since their real storage was actually a portion of the virtual storage assigned from the VM host's real storage as required. The virtual addresses of each V=V guest translated to real addresses which were different from its virtual addresses, unlike the V=R preferred guest. The result was better performance for the one preferred guest due to assigning the lowest address part of MS to the preferred guest operating system beginning at absolute address zero. Because the V=R guest had equal real and virtual addresses, I/O programs using real addresses would access the correct locations in real main storage even when they were translated, and this enabled the V=R guest to handle its own I/O starts and I/O interruptions and its own address translations, without host intervention, and the V=R guest had a performance efficiency approaching its standalone performance efficiency (i.e. when it was the only programming system in the data processing system).
But during V=V guest program execution, whenever the guest needs a main storage allocation, or requires I/O and tries to issue a start I/O instruction (SIO, SIOF or SSCH), the guest is not permitted to do so, because the guest's I/O program real addresses are not equal to the guest's assigned absolute addresses in MS. Hence, the guest's channel programs will not operate when the VM assigned location of the guest is not in the MS area addressed by the channel program.
Because of these storage address problems, the V=V guest's program requests for storage and SIO, SIOF or SSCH instructions were intercepted by the VM host control program, which then adjusted the I/O program and storage addresses for the guest so they would address locations within the MS area assigned for use by the guest.
To do the guest I/O in S/370, the VM host first must assign a part of the host's virtual storage space to each V-V guest. A guest needing to access guest real addresses (as occur for MS requests by guest I/O operations) must use each guest real address as a host virtual address, which must be translated (using the host segment and page tables) to a host absolute address. Consequently MS is accessed at this host absolute address in response to the guest's I/O channel request. Therefore, each guest I/O request accesses MS at a location obtained from a host address translation. Another page address translation is needed whenever the I/O requests cross a page boundary in MS.
Once a CPU page is translated (non-I/O address), the resulting page frame address is stored in a TLB in the CPU, which is referenced to avoid repeating the translation process for the same guest page. The address translation process previously used for V=V and V=R guests is explained in detail in U.S. Pat. No. 4,456,954 to Bullions et al. Other U.S. patent applications providing relevant prior art are Ser. No. 947,350, by P. H. Gum et al, filed Dec. 29, 1986, entitled "Selective Guest System Purge Control" and Ser. No. 651,491, by H. R. Brandt et al, filed Sept. 17, 1984, entitled "Fast Two-Level Dynamic Address Translation Method and Means", both of which are assigned to the same assignee as the subject application.
If the referenced guest storage is not resident in the host real storage, a page in operation must be performed.
When a page in guest storage is to be referenced for an I/O channel operation, its page frame must be "locked" in host storage for the duration of the I/O accessing operation.
The I/O operation initially finds a guest's S/370 channel program location by going to location 72 in the guest's PSA page frame (i.e. the guest's page zero) to obtain the channel address word (CAW) which contains the address of the first channel command word (CCW) in the guest's channel program. To start an I/O operation in S/370XA, a guest issues a start subchannel (SSCH) instruction which addresses an operation request block (ORB) located anywhere in MS to address the first CCW of a channel program. In order to relocate the channel program, VM translates the address of the obtained first CCW to an absolute address in MS. Then the VM host examines the channel program to determine which pages in the guest's area are needed for the data being transferred, and VM fixes page frames for these pages in preparation for the I/O data transfer. Then the VM host copies the channel program into the host's MS area and changes the addresses within its CCWs to the absolute addresses which are required by the I/O channel hardware to be accessed in the newly fixed page frames in the guest area for the I/O data transfer. Finally, the VM host starts execution of the I/O channel program it built in the VM host area to perform the I/O transfer for the guest. Hence, it is apparent that guest I/O operation under VM involves a significant amount of programming overhead before the channel program can start to run, which is overhead that does not exist when the guest program is executed as a program in the original MS area for which it was written; which is done for a V=R guest, or in a standalone environment where it is the only program in a system and has all of the resources in the system for its use.
In S/370XA, the host finds the guest channel program location by obtaining the guest start subchannel (SSCH) instruction and operation request block (ORB) operand address.
The added VM host overhead for V=V guest's to obtain real address relocation of guest I/O channel programs has been recognized as burdensome for production guests, i.e. highly active guests. The limited solution for the one V=R preferred guest, loaded at absolute address zero, is not available for other guests. As long as there is no need for more than one highly efficient guest, the conventional VM preferred guest solution is adequate.
The added burden of the conventional VM handling of non-preferred V=V guests becomes significant for non-preferred guests that have a large amount of I/O activity. The result is that V=V guest programs having large amounts of I/O activity cannot be efficiently handled by VM.
This conventional VM limitation occurs whether VM is being run on a UP or an MP, because the main storage of either has a single expanse of absolute addresses (absolute addresses are the same as real addresses in a UP). In some MPs, one of the plural CPUs may be dedicated to execute the preferred guest in the system. Then another CPU in the MP may execute the non-preferred guests.
It should be understood that a guest may be an operating system representing UP or MP operations. Hence, a guest may be a UP guest or a MP guest. An MP guest can be executed on a real UP system as well as on a real MP system.
In a real MP or UP, address translation requires a page frame assignment to each page of program requested logical addresses in order to translate program virtual addresses to main storage real addresses. In a MP, program requested real addresses are changed to absolute addresses by CPU hardware prefixing (which is not apparent to the user). Absolute addressing is required in a MP because each of the plural CPUs has its own control area (called a PSA, prefix save area) and each PSA is located at real address zero. MP main storage has only one physical address of zero (called absolute address zero); and each CPU's real address zero is located at a prefixed non-zero absolute address, so that the different CPU PSA (all at the real address of zero) are relocated at a different non-overlapping absolute address locations in the single range of absolute addresses in the MP machine's main storage. Normally the page frame located at absolute address zero is not used as a prefixed PSA page. Thus, the CPU real addresses provided for accessing the PSA pages are prefixed differently in the MS range of absolute addresses, but all other CPU real addresses provided for accessing main storage are the same as their corresponding absolute addresses. In a UP, all real addresses may also be absolute addresses. However, prefixing has been used in UPs (as well as in MPs) for locating the host PSA page outside of the area allocated to a V=R guest which includes the first page frame in MS.
A solution to the I/O burden for a MP using a programming system designed for UP operation is to redesign the program in read-only form with proper internal programmed locks so that a particular I/O program can only execute on one at a time of the plural CPUs. This involves having only one copy of the program in MS beginning at absolute address zero, and except for its locked parts, this copy is simultaneously and independently being executed on the plural CPUs with necessary program coordination being maintained by the internal programmed locks.
Unfortunately, the redesign of a complex UP program for MP operation is expensive and takes a long period of time to complete in a reasonably error free manner. Many existing uniprocessor programming systems (UPSs) were written with real addressing (rather than virtual addressing). These programs generally use 24 bit real addresses, and have very large amounts of I/O activity. They may not be designed with the internal locks required to maintain the integrity of the program on a MP.
Since there is only one set of absolute addresses in a MP between 0 and a maximum value (that can vary with each MP machine, for example 64 MB), only one copy of a program (written to start at address zero using real addresses) can be located in the main storage, when the program, or any part of it such as its I/O channel programs, is not relocatable. This may be the case with existing programs written with real addressing for execution on a UP. Inter-CPU locks (that may be needed in the programs to enable their use on a MP) may not exist in such a program.
Consequently in the prior art: (1) a program written for UP use may not be executable on more than one CPU of a MP, because of a lack of programmed locks in its internal program routines to assure against failure of execution caused by plural CPU contention for its writable parts; (2) indirect execution of plural copies of such programs on plural CPUs under VM (although operable) becomes economically impractical when they have very high I/O activity because of the VM host's I/O simulation burden, and (3) it is very expensive to rewrite such programs for plural CPU operation, since it would take a substantial amount of time and testing, although this can be done by those presently skilled in the MP programming arts.
As a result, the most efficient processing for such UP designed programs has been limited to single CPU operation in a standalone UP, even though faster and more reliable MPs are available.
It is of course possible to utilize separate standalone CPUs with independent main storages, but generally MP hardware provides a less expensive package for a given amount of processing power because of the sharing of the physical components of the system among the simultaneous programs being executed.
U.S. Pat. No. 4,564,903 to Guyette et al and assigned to the same assignee as this application, discloses and claims a unique multiprocessing environment that enables plural copies of the same uniprocessor programming system (UPS) not designed to run on a MP system, to execute simultaneously on a MP. That patent provides a hypervisor type of control program (called a partitioned multiprocessing program, PMP) that enables the simultaneous execution in a MP by its plural CPUs of respective copies of a UPS in the MPs main storage (MS) with the capability of sharing system resources, including a single I/O data base, e.g. on plural DASDs. Copies of the same UPS program are loaded into different areas in the MPs main storage (MS). Each area begins at a different MS location and comprises a contiguous byte area. PMP can provide any CPU in the MP with an affinity to a particular copy of the UPS in MS. The plural CPUs executing the different copies of the UPS run independently of each other, but they may share I/O devices. Such system runs with a virtual machine (VM) type of job entry and task dispatching control programming system preferably designed to operate on CPUs having an architecture disclosed in the "IBM System/370 Extended Architecture (S/370-XA) Principles of Operation" (IBM Publication No. SA22-7085-0), in the "IBM Assists for MVS/XA" (IBM Publication No. SA22-7092-0), and in an article entitled "System/370 Extended Architecture: Design Considerations" in the May 1983 IBM Journal of Research and Development, Volume 27, Number 3 on pages 198 to 205 by A. Padegs. These publications include SIE (start interpretive execution) architecture.
Program jobs are entered for execution by a respective UPS copy, and the jobs are controlled under PMP as tasks of the respective UPS guest.
PMP ensures MP integrity: (1) in its assignments of all tasks in the MP, and (2) in its assignment of each I/O device to any UPS operation. PMP ensures integrity of the physical and logical I/O paths in the MP shared by the multiple UPS copies by maintaining I/O queues to hold pending I/O operations issued to I/O units which are busy performing I/O operations requested by the different UPS copies. When any required I/O unit becomes free, PMP can issue the next pending I/O operation from the queues.
The PMP hypervisor is initially program loaded into MS, preferably at the high address end of MS. Thereafter, copies of the UPS can be loaded into the different MS areas. A next copy can be loaded into MS at any time by a log on process.
The size of MS must equal or exceed the number of areas provided for the respective copies of the same UPS plus the PMP hypervisor, and any other program areas. The UPS program uses an effective address range which only covers its respective area in MS, and its address range may be insufficient to address the absolute address size of MS. For example, the UPS might use 24 bit addresses, but MS absolute addresses may exceed 24 bit addressing, e.g. requiring 26 bits. PMP hypervisor runs independently on any one or more of the CPUs in the MP to dispatch all jobs on each respective CPU, including the dispatching of both UPS and non-UPS work. Non-UPS work may be executed on a CPU if a UPS has not been assigned to it, or the assigned UPS is not ready for execution. Non-UPS work may be VM work in which case PMP passes control to the VM like PMP control program, which then controls the CPU and I/O execution of VM work in the conventional VM manner. PMP therefore passes UPS and VM work to the CPUs for execution. Under PMP control, some or all of the CPUs may be dedicated solely to UPS work. VM work may be done on any CPU not dedicated solely to UPS work or on a CPU dedicated to VM work.
To enable I/O addresses in any UPS copy (i.e. I/O real addresses) to be handled in a simple manner that translates them to MS absolute addresses, each zone begins at a boundary byte address which is an integral power of the radix 2, i.e. 2 to the n power (2**n). The first UPS region may begin at absolute address zero. The 2**n byte boundary enables a MS absolute address to be easily generated by logically ORing a representation of the 2**n-boundary value and the low order part of the UPS I/O address to form a 31 bit absolute address. More general I/O address translation means (more complex and costly) may be used to enable the zone boundaries to be located more flexibly.
The lowest address in each MS area begins with the PSA page of the respective guest UPS copy which may be structured in accordance with S/370 architecture and begins with the guest's absolute address zero. The contents of each guest's PSA page is simulated, i.e. entirely written in with programmed instructions rather than by CPU hardware actions.
However, each CPU in the system also has a hardware controlled PSA page which receives hardware inputted contents, e.g. for hardware interrupts. Each CPU PSA page is preferably located in the PMP area of MS by the content of the respective CPUs prefix register. The new PSWs in the CPU PSA page point to routines in the PMP host area for the handling of respective hardware interrupts.
PMP is the host control program which enables the UPS copies to be executed as emulated guests on the respective CPUs by putting each CPU into emulation state for executing a UPS guest program. One method of having PMP put a CPU operating in 370XA mode into emulation state is to have PMP execute a SIE (start interpretive execution) instruction, which causes the CPU to emulate the S/360 or S/370 or S/370XA architecture. The SIE instruction is explained in an article in the Proceedings of the Spring Meeting 1986, SHARE European Association, Heidelberg, West Germany, Apr. 06-11, page 531, "SIE Architecture", by P. H. Gum, and SIE is discussed in U.S. Pat. No. 4,456,954, issued to R. J. Bullions et al and assigned to the same assignee as the subject application.
Each UPS copy in MS is handled by MP as a separate guest which may be emulated on a different one of the real CPUs in the MP by PMP executing a SIE instruction on each of such real CPUs to put it into emulation state to emulate thereon a logical CPU whenever a real CPU is available to execute a ready UPS guest not dedicated to another CPU.
The UPS guest emulation state (i.e. the dispatch of a logical CPU on any real CPU) is temporarily ended whenever a hardware interruption occurs or the UPS guest executes either: (1) a UPS start I/O instruction, or (2) a UPS instruction (e.g. set system mask) that sets on an I/O enablement field in the guest's current program status word (PSW). When the emulation state is exited, CPU control is transferred from the UPS guest to the PMP host, and then the host respectively: (1) starts the requested I/O device, (2) simulates a special instruction, e.g. SSM, causing interruption, or (3) handles the I/O or other interruption causing the exit from emulation mode. Then the PMP host again puts the CPU back into the emulation state when the CPU again becomes available to continue execution by a ready UPS guest.
European patent application No. 0 171 475 by R. S. Lent et al, recently published, discloses a host computer system (UP or MP) in which each central processor and each I/O processor (built to S/370 architecture) is provided with a hardware element called a logical processor facility (LPF). (This European application is believed to represent the currently marketed Amdahl MDF system.) In the application, the LPF has user and system states; and each state has problem and supervisor modes. The LPF hardware supports simultaneous operation of plural operating system programs (SCPs) located in different domains in main storage. When in system state, the CPU or I/O processor is working for the chief SCP; and when in user state, the central or I/O processor is working for one of the user SCPs. The chief SCP controls the dispatching of all user SCPs on the central processors. The user SCPs may be written for other environments and are unaltered for operation on the host computer system. I/O channel addresses are relocated into the domain of the SCP that started the channel, and a lock bit controls whether the I/O channel is to be controlled by the user SCP or by the chief SCP. LPF registers contain the domain number and the domain CPU number. Interrupt router hardware directs each I/O interrupt to its required central processor by comparing a domain number with the I/O interrupt and the domain CPU number in the central processor, and if they match, the router sends the I/O interrupt to that CPU. Also, a user supervisor state bit in the I/O processor and each central processor are compared, and if both are in user state the processor processes the interrupt; but if the I/O processor is in user state and the central processor is in supervisor state then the router will not transfer the interrupt until the required processor returns to user state. LPF permits only the central processor that started the I/O device causing the interrupt to receive and handle the interrupt, which prevents the LPF host from dispatching a guest on any available processor, and the LPF host can only dispatch a guest on the same central processor that issued the related start I/O instruction and therefore must await its availability, thereby preventing the LPF system from providing its processors with dispatching flexibility in being able to dispatch a guest on the first processor that becomes available.